25 research outputs found
Recommended from our members
Neural Approaches to Feedback in Information Retrieval
Relevance feedback on search results indicates users\u27 search intent and preferences. Extensive studies have shown that incorporating relevance feedback (RF) on the top k (usually 10) ranked results significantly improves the performance of re-ranking. However, most existing research on user feedback focuses on words-based retrieval models. Recently, neural retrieval models have shown their efficacy in capturing relevance matching in retrieval but little research has been conducted on neural approaches to feedback. This leads us to study different aspects of feedback with neural approaches in the dissertation.
RF techniques are seldom used in real search scenarios since they can require significant manual efforts to obtain explicit judgments for search results. However, with mobile or voice-based intelligent assistants being more popular nowadays, user feedback of result quality could be collected potentially during their interactions with the assistants. We study both positive and negative RF to refine the re-ranking performance. Positive feedback aims to find more relevant results given some known relevant results while negative feedback targets identifying the first relevant result. In most cases, it is more beneficial to find the first relevant result compared with finding additional relevant results. However, negative feedback is much more challenging than positive feedback since relevant results are usually similar while non-relevant results could vary considerably.
We focus on the tasks of text retrieval and product search to study the different aspects of incorporating feedback for ranking refinement with neural approaches. Our contributions are: (1) we show that iterative relevance feedback (IRF) is more effective than top-k RF on answer passages and we further improve IRF with neural approaches; (2) we propose an effective RF technique based on neural models for product search; (3) we study how to refine re-ranking with negative feedback for conversational product search; (4) we leverage negative feedback in user responses to ask clarifying questions in open-domain conversational search. Our research improves retrieval performance by incorporating feedback in interactive retrieval and approaches multi-turn conversational information-seeking tasks with a focus on positive and negative feedback
Learning a Deep Listwise Context Model for Ranking Refinement
Learning to rank has been intensively studied and widely applied in
information retrieval. Typically, a global ranking function is learned from a
set of labeled data, which can achieve good performance on average but may be
suboptimal for individual queries by ignoring the fact that relevant documents
for different queries may have different distributions in the feature space.
Inspired by the idea of pseudo relevance feedback where top ranked documents,
which we refer as the \textit{local ranking context}, can provide important
information about the query's characteristics, we propose to use the inherent
feature distributions of the top results to learn a Deep Listwise Context Model
that helps us fine tune the initial ranked list. Specifically, we employ a
recurrent neural network to sequentially encode the top results using their
feature vectors, learn a local context model and use it to re-rank the top
results. There are three merits with our model: (1) Our model can capture the
local ranking context based on the complex interactions between top results
using a deep neural network; (2) Our model can be built upon existing
learning-to-rank methods by directly using their extracted feature vectors; (3)
Our model is trained with an attention-based loss function, which is more
effective and efficient than many existing listwise methods. Experimental
results show that the proposed model can significantly improve the
state-of-the-art learning to rank methods on benchmark retrieval corpora
Unbiased Learning to Rank with Unbiased Propensity Estimation
Learning to rank with biased click data is a well-known challenge. A variety
of methods has been explored to debias click data for learning to rank such as
click models, result interleaving and, more recently, the unbiased
learning-to-rank framework based on inverse propensity weighting. Despite their
differences, most existing studies separate the estimation of click bias
(namely the \textit{propensity model}) from the learning of ranking algorithms.
To estimate click propensities, they either conduct online result
randomization, which can negatively affect the user experience, or offline
parameter estimation, which has special requirements for click data and is
optimized for objectives (e.g. click likelihood) that are not directly related
to the ranking performance of the system. In this work, we address those
problems by unifying the learning of propensity models and ranking models. We
find that the problem of estimating a propensity model from click data is a
dual problem of unbiased learning to rank. Based on this observation, we
propose a Dual Learning Algorithm (DLA) that jointly learns an unbiased ranker
and an \textit{unbiased propensity model}. DLA is an automatic unbiased
learning-to-rank framework as it directly learns unbiased ranking models from
biased click data without any preprocessing. It can adapt to the change of bias
distributions and is applicable to online learning. Our empirical experiments
with synthetic and real-world data show that the models trained with DLA
significantly outperformed the unbiased learning-to-rank algorithms based on
result randomization and the models trained with relevance signals extracted by
click models
A Transformer-based Embedding Model for Personalized Product Search
Product search is an important way for people to browse and purchase items on
E-commerce platforms. While customers tend to make choices based on their
personal tastes and preferences, analysis of commercial product search logs has
shown that personalization does not always improve product search quality. Most
existing product search techniques, however, conduct undifferentiated
personalization across search sessions. They either use a fixed coefficient to
control the influence of personalization or let personalization take effect all
the time with an attention mechanism. The only notable exception is the
recently proposed zero-attention model (ZAM) that can adaptively adjust the
effect of personalization by allowing the query to attend to a zero vector.
Nonetheless, in ZAM, personalization can act at most as equally important as
the query and the representations of items are static across the collection
regardless of the items co-occurring in the user's historical purchases. Aware
of these limitations, we propose a transformer-based embedding model (TEM) for
personalized product search, which could dynamically control the influence of
personalization by encoding the sequence of query and user's purchase history
with a transformer architecture. Personalization could have a dominant impact
when necessary and interactions between items can be taken into consideration
when computing attention weights. Experimental results show that TEM
outperforms state-of-the-art personalization product retrieval models
significantly.Comment: In the proceedings of SIGIR 202
CIR at the NTCIR-17 ULTRE-2 Task
The Chinese academy of sciences Information Retrieval team (CIR) has
participated in the NTCIR-17 ULTRE-2 task. This paper describes our approaches
and reports our results on the ULTRE-2 task. We recognize the issue of false
negatives in the Baidu search data in this competition is very severe, much
more severe than position bias. Hence, we adopt the Dual Learning Algorithm
(DLA) to address the position bias and use it as an auxiliary model to study
how to alleviate the false negative issue. We approach the problem from two
perspectives: 1) correcting the labels for non-clicked items by a relevance
judgment model trained from DLA, and learn a new ranker that is initialized
from DLA; 2) including random documents as true negatives and documents that
have partial matching as hard negatives. Both methods can enhance the model
performance and our best method has achieved nDCG@10 of 0.5355, which is 2.66%
better than the best score from the organizer.Comment: 5 pages, 1 figure, NTCIR-1